Sourcerer: A Search Engine for Open Source Code

sourcerer is a search engine for open source code that extracts fine-grained structural information from the code. This information is used both to implement a basic notion of code rank and to enable search forms that go beyond conventional keyword-based searches. sourcerer supports two types of searches: (1) implementations, and their use; and (2) program structures. Several schemes were compared for ranking the results of code search. Results are reported involving 1,555 open source Java projects, corresponding to 254 thousand classes and 17 million LOCs. Of the schemes compared, the scheme that produced the best search results was one consisting of a combination of (a) the standard TF-IDF technique over Fully Qualified Names (FQNs) of code entities, with (b) a “boosting” factor for terms found towards the right-most handside of FQNs, and (c) a composition with a graph-rank algorithm that identifies popular classes.